Optimistic Planning for Markov Decision Processes

International audienceThe reinforcement learning community has recently intensified its interest in online planning methods, due to their relative independence on the state space size. However, tight near-optimality guarantees are not yet available for the general case of stochastic Markov decision processes and closed-loop, state-dependent planning policies. We therefore consider an algorithm related to AO* that optimistically explores a tree representation of the space of closed-loop policies, and we analyze the near-optimality of the action it returns after n tree node expansions. While this optimistic planning requires a finite number of actions and possible next states for each transition, its asymptotic performance does not depend directly on these numbers, but only on the subset of nodes that significantly impact near-optimal policies. We characterize this set by introducing a novel measure of problem complexity, called the near-optimality exponent. Specializing the exponent and performance bound for some interesting classes of MDPs illustrates the algorithm works better when there are fewer near-optimal policies and less uniform transition probabilities

Development and initial validation of the obsessive beliefs questionnaire and the interpretation of intrusions inventory

In 1995 the Obsessive Compulsive Cognitions Working Group initiated a collective process to develop two measures of cognition relevant to current cognitive-behavioural models of OCD. An earlier report (Behav. Res. Therapy, 35 (1997) 667) describes the original process of defining relevant domains. This article describes the subsequent steps of the development and validation process: item generation, scale reduction, and initial examination of reliability and validity. Two scales were developed. The Obsessive Beliefs Questionnaire consists of 87 items representing dysfunctional assumptions covering six domains: overestimation of threat, tolerance of uncertainty, importance of thoughts, control of thoughts, responsibility, and perfectionism. The Interpretation of Intrusions Inventory consists of 31 items that refer to interpretations of intrusions that have occurred recently. Three of the above domains are represented: importance of thoughts, control of thoughts, and responsibility. The item reduction and validation analyses were conducted on clinical and non-clinical samples from multiple sites. Initial examination of reliability and validity indicates excellent internal consistency and stability and encouraging evidence of validity. However, high correlations indicating overlap between some of the scales, particularly importance of thoughts, control of thoughts, and responsibility will need to be addressed in subsequent empirical and theoretical investigations

The FreeMABSys project and the BLAD libraries

International audienceNo abstrac

Metformin in women with type 2 diabetes in pregnancy (MiTy): a multicentre, international, randomised, placebo-controlled trial

Background: Although metformin is increasingly being used in women with type 2 diabetes during pregnancy, little data exist on the benefits and harms of metformin use on pregnancy outcomes in these women. We aimed to investigate the effects of the addition of metformin to a standard regimen of insulin on neonatal morbidity and mortality in pregnant women with type 2 diabetes. Methods: In this prospective, multicentre, international, randomised, parallel, double-masked, placebo-controlled trial, women with type 2 diabetes during pregnancy were randomly assigned from 25 centres in Canada and four in Australia to receive either metformin 1000 mg twice daily or placebo, added to insulin. Randomisation was done via a web-based computerised randomisation service and stratified by centre and pre-pregnancy BMI (<30 kg/m2 or ≥30 kg/m2) in a ratio of 1:1 using random block sizes of 4 and 6. Women were eligible if they had type 2 diabetes, were on insulin, had a singleton viable pregnancy, and were between 6 and 22 weeks plus 6 days' gestation. Participants were asked to check their fasting blood glucose level before the first meal of the day, before the last meal of the day, and 2 h after each meal. Insulin doses were adjusted aiming for identical glucose targets (fasting glucose <5·3 mmol/L [95 mg/dL], 2-h postprandial glucose <6·7 mmol/L [120 mg/dL]). Study visits were done monthly and patients were seen every 1–4 weeks as was needed for standard clinical care. At study visits blood pressure and bodyweight were measured; patients were asked about tolerance to their pills, any hospitalisations, insulin doses, and severe hypoglycaemia events; and glucometer readings were downloaded to the central coordinating centre. Participants, caregivers, and outcome assessors were masked to the intervention. The primary outcome was a composite of fetal and neonatal outcomes, for which we calculated the relative risk and 95% CI between groups, stratifying by site and BMI using a log-binomial regression model with an intention-to-treat analysis. Secondary outcomes included several relevant maternal and neonatal outcomes. The trial was registered with ClinicalTrials.gov, NCT01353391. Findings: Between May 25, 2011, and Oct 11, 2018, we randomly assigned 502 women, 253 (50%) to metformin and 249 (50%) to placebo. Complete data were available for 233 (92%) participants in the metformin group and 240 (96%) in the placebo group for the primary outcome. We found no significant difference in the primary composite neonatal outcome between the two groups (40% vs 40%; p=0·86; relative risk [RR] 1·02 [0·83 to 1·26]). Compared with women in the placebo group, metformin-treated women achieved better glycaemic control (HbA1c at 34 weeks' gestation 41·0 mmol/mol [SD 8·5] vs 43·2 mmol/mol [–10]; 5·90% vs 6·10%; p=0·015; mean glucose 6·05 [0·93] vs 6·27 [0·90]; difference −0·2 [–0·4 to 0·0]), required less insulin (1·1 units per kg per day vs 1·5 units per kg per day; difference −0·4 [95% CI −0·5 to −0·2]; p<0·0001), gained less weight (7·2 kg vs 9·0 kg; difference −1·8 [–2·7 to −0·9]; p<0·0001) and had fewer caesarean births (125 [53%] of 234 in the metformin group vs 148 [63%] of 236 in the placebo group; relative risk [RR] 0·85 [95% CI 0·73 to 0·99]; p=0·031). We found no significant difference between the groups in hypertensive disorders (55 [23%] in the metformin group vs 56 [23%] in the placebo group; p=0·93; RR 0·99 [0·72 to 1·35]). Compared with those in the placebo group, metformin-exposed infants weighed less (mean birthweight 3156 g [SD 742] vs 3375 g [742]; difference −218 [–353 to −82]; p=0·002), fewer were above the 97th centile for birthweight (20 [9%] in the metformin group vs 34 [15%] in the placebo group; RR 0·58 [0·34 to 0·97]; p=0·041), fewer weighed 4000 g or more at birth (28 [12%] in the metformin group vs 44 [19%] in the placebo group; RR 0·65 [0·43 to 0·99]; p=0·046), and metformin-exposed infants had reduced adiposity measures (mean sum of skinfolds 16·0 mm [SD 5·0] vs 17·4 [6·2] mm; difference −1·41 [–2·6 to −0·2]; p=0·024; mean neonatal fat mass 13·2 [SD 6·2] vs 14·6 [5·0]; p=0·017). 30 (13%) infants in the metformin group and 15 (7%) in the placebo group were small for gestational age (RR 1·96 [1·10 to 3·64]; p=0·026). We found no significant difference in the cord c-peptide between groups (673 pmol/L [435] in the metformin group vs 758 pmol/L [595] in the placebo group; p=0·10; ratio of means 0·88 [0·72 to 1·02]). The most common adverse event reported was gastrointestinal (38 events in the metformin group and 38 events in the placebo group). Interpretation: We found several maternal glycaemic and neonatal adiposity benefits in the metformin group. Along with reduced maternal weight gain and insulin dosage and improved glycaemic control, the lower adiposity and infant size measurements resulted in fewer large infants but a higher proportion of small-for-gestational-age infants. Understanding the implications of these effects on infants will be important to properly advise patients who are contemplating the use of metformin during pregnancy.The trial was funded by the Canadian Institutes of Health Research, the Lunenfeld-Tanenbaum Research Institute, Toronto, ON, Canada, and the Department of Medicine, University of Toronto, Toronto, ON, Canada